Introduction

Education and training are key components of any rehabilitation program. While such services are ideally delivered in person, cost, time and travel demands often prevent clinicians from offering this level of personalized interaction. Mobility is the most prevalent area of impairment among Canadian older adults and hence a key target for such programs.  Manual wheelchairs (MWCs) are a commonly used assistive device, but merely acquiring a MWC does not guarantee satisfactory performance of desired activities: In Canada, over 90% of older MWC users experienced restricted performance in at least one major life activity (Statistics Canada, 2008). A structured education program delivered in-person, such as the Wheelchair Skills Training Program (WSTP), can produce significant improvement in skill capacity among adult MWC users (for example, see (Best, Kirby, Smith, & MacLeod, 2005)); however, few older adults receive such training because of resource limitations.

Alternatively, monitored or self-managed home rehabilitation training programs have been effective and can be done at convenient times in a private and familiar setting without the time, effort or expense of travel (for example, see (Jette, et al., 1998)).  This paper presents the Monitored Education and Training Tablet App (METTA), a platform for video-based training programs delivered at the homes of trainees on an Android tablet and asynchronously monitored by trainers through a website.  METTA was developed concurrently with EPICWheelS, a MWC training program currently undergoing a randomized controlled trial (Giesbrecht, Miller, Eng, Mitchell, Woodgate, & Goldsmith, 2013); however, METTA could support other similar training programs and so in this paper we describe the design of and technology behind it.

The rapidly expanding field of mHealth studies the use of mobile technologies like smartphones and tablets for delivery of health interventions.  In (Klasnja & Pratt, 2012) the authors identify five key mHealth intervention strategies: increasing the availability of health information, tracking health information, involving the healthcare team, leveraging social influence and utilizing entertainment.  Our platform currently supports the first three of these by providing training videos in a format that can be watched almost anywhere (eg: while sitting in the MWC), tracking training time and video usage, and delivering that information to a trainer respectively.  Although there are a plethora of mHealth applications available (for example, see the extensive bibliographies of (Klasnja & Pratt, 2012) and (Liu, Zhu, Holroyd, & Seng, 2011)), the only similar system we have discovered is (StrokeLink), a commercial product which is used to provide a training program and other information in a variety of media formats (including videos) on an iPad to stroke survivors; however, we have been unable to find any formal evaluation of its efficacy or descriptions of its internals.  An iPhone based tool for tracking and analyzing rehabilitation tasks is presented in (Zerin, O'Brien, Ahamed, & Smith, 2013), but it is aimed at practitioners rather than patients.

An extended version of this paper containing sample screenshots will be made available through UBC’s cIRcle digital repository (https://circle.ubc.ca).

Approach

The EPICWheelS Training Program

The purpose of the Enhancing Participation In the Community by improving Wheelchair Skills (EPICWheelS) project is to develop a home training program for older adult novice MWC users monitored by an expert trainer and delivered on a tablet computer.  Although informed by the WSTP, the EPICWheelS training program does not cover precisely the same set of skills, and is designed to address the specific learning needs of older adults and the challenges of video-based training outside the clinic.  Trainees and their caregivers are introduced to their tablet by trainers at the beginning of the program, and are asked to watch videos and practice the skills almost daily over a multi-week period.  Trainees are given a wireless hotspot for internet connectivity and an adjustable platform that can be secured around their thighs onto which the tablet can be attached with Velcro.

No existing tablet app could be found to provide the desired features, so a Participatory Action Design (PAD) process was undertaken (Ding, Coooper, & Pearlman, 2007) to design the training content and tablet app together.  PAD emphasizes the substantive contributions that intended users can make during a design process.  Two focus groups composed of MWC users provided feedback on an initial prototype; after refinement of the prototype six more focus groups (two each consisting of MWC users, their caregivers and clinicians) provided additional feedback.  The resulting version 1 was subjected to critical evaluation by an MWC user and a clinician, and then full multi-week pilot tests were performed by two MWC users.  Based on feedback from those rounds of testing, a version 2 was developed and is currently being used in the EPICWheelS RCT.  The trainers’ website interface was not taken through the PAD process, although its contents (the data collected by the app) were part of the PAD process, and the project team already includes clinicians.

Training Program Roles

Early in the design process we determined that users of the system would interact with it in a number of different roles, each of which needs access to certain information and certain capabilities.  Note that individuals can play multiple roles.

Trainees: The MWC users undertaking the training program (and at this stage enrolled in the EPICWheelS study).  They will access the training videos through a personal Android tablet.  They can communicate with their trainers through the messaging system.

Caregivers: For safety reasons, Trainees are encouraged to have a caregiver participate in home training activities, and a spotters strap is provided to reduce the chances of tips when practicing the more advanced skills.  It was decided that the caregivers could share trainees’ access to the training program, although future versions might add a separately tracked identity for caregivers.

Trainers: Clinicians who monitor their trainees’ training activities and progress, communicate with their trainees through the messaging system, and enable / disable a custom subset of the skills to create a tailored training regime for each trainee.  They will access the system through a web-based interface using a desktop or laptop computer.  Normally trainers will monitor multiple trainees while a trainee will have only one trainer; however, it is possible to assign multiple trainers to a single trainee (for example, to cover trainer vacations).

Study / account managers: The member(s) of the EPICWheelS research team running the RCT.  They will create accounts for new trainees and trainers, and assign trainees to trainers.  They will install and configure the app for each new trainee.  Because tablets are reused during the RCT, they will archive and clear trainee data from returned tablets.  They can monitor and download records of trainee training activities for analysis as part of the study through a web-based interface like that used by the trainers.

Content designers:  Domain experts who create the training videos, choose their properties, and structure the hierarchy in which they are presented to trainees.  They create the training program through a separate web-based interface, and the result is a small set of files which are installed on the trainees’ tablets.

System administrators: Individuals with sufficient information technology (IT) experience to create and maintain a Linux machine which serves the websites and records trainee data.  We use standard web and network protocols so that this role does not need any application specific training; however, it is still important to have somebody who can, for example, restart the web server in case of unexpected failures.

In the next few sections we describe the experience of interacting with the system for users in the four key roles.  Technical details of METTA are then described.

Trainee Interaction

Although the system can be installed on a tablet as a standard app, for the EPICWheelS trial we decided to use a custom launcher so that the app starts automatically whenever the tablet is turned on.

The training content is arranged as a hierarchical tree.  Each non-leaf node in the tree is presented to the trainee as a menu containing buttons for each child of that node.  The root node of the tree corresponds to the app’s home screen, which shows the first level of submenu buttons as well as some summary information about the trainees’ progress toward various training goals and awards.  All other non-leaf nodes show buttons for each of that node’s children, the total number of minutes spent watching leaf children of this menu, and a button to return to the parent node.  Each button shows the name of the corresponding node and the amount of time spent in that node (and all child nodes).  To improve motivation, trainees can collect two types of awards: checkmarks and stars.  For the buttons corresponding to leaf nodes, a checkmark is shown if the corresponding video has been at least partially viewed, and a star is shown if the time spent watching that video is roughly as long as the video itself (there is no direct way to determine whether the video has been watched from start to finish).  For the buttons corresponding to non-leaf nodes, a checkmark / star appears if all children of the corresponding submenu have a checkmark / star respectively.

Each leaf node in the tree is a video.  The video is shown on the left side of the screen and can be controlled through buttons on the right side of the screen.  In addition to the video control buttons, there is a start/stop button that trainees can use like a stopwatch to record time spent practicing the task associated with a video when the video is not playing.

Before viewing any training videos, each trainee must watch a collection of safety videos.  To further minimize risk, entry to the menus associated with some of the more advanced skills triggers a popup window that asks trainees to confirm that they are familiar with the safety videos (and gives them the option to immediately watch them again).

Trainees can also record voice or video messages and send them to their trainer, or receive voice messages from their trainer.  The message menu can be accessed from any other menu in the system.

For the EPICWheelS trial, some additional daily survey questions are posed to trainees.  Trainees are asked whether they suffered tips or falls since their last practice session, whether they performed any extra practice without the tablet, and if so, how long they spent in that extra practice.

Trainer Interaction

Three tabs are available for each trainee at the trainers’ website.  The first shows information about that trainee’s activities in the form of several tables; for example, summary statistics about the amount of time spent on various categories of interaction with the system.  The data from each table can be downloaded as a comma separated value file for further analysis (eg: with spreadsheet software).  The second tab allows the trainer to access the trainee’s voice and video messages and reply with voice messages.  The third tab allows the trainer to enable / disable specific nodes in the training content tree for that particular trainee.

Study / Account Manager Interaction

When a tablet is returned, the process of archiving and clearing data from the previous trainee and installing and configuring the system for a new trainee is complicated by data confidentiality concerns, various constraints imposed by Android, and the sheer size of the video content (more than four gigabytes in the current version); consequently, this step has no simple tablet or web interface.  That said, we assume that the user undertaking this role has no IT experience beyond that needed to work with a standard Mac or Windows machine.  A detailed checklist was developed to guide managers through the series of file transfers, menu selections and scripts to accomplish these tasks.  The manager also has access to a web interface identical to the trainers’ interface, but with all trainees included, in order to download any data needed for study analysis.

Content Designer Interaction

The creation of the content hierarchy and filling it with videos occurs before the tablets are configured, so the content designer’s web site is completely separate from the site used by the trainers and managers.  On this site the content designer can add new nodes (either videos or menus) to the hierarchy, as well as edit or delete existing nodes.  All nodes have a name and a description associated with them; they may optionally specify which videos must be watched before the node may be accessed (eg: safety videos).  Video nodes require the designer to upload a video (in MP4 format) and to specify how time spent watching that video should be classified during training.  At present there are two categories of videos—“activity” and “instructional”—and total time spent viewing videos from each of these categories is shown separately in the trainer’s summary statistics table.

Technical Details

The tablet can determine the time at which each button is pressed and thereby estimate how long the trainee spends watching videos.  If trainees conscientiously use the start/stop button beside each video to indicate the time they spend practicing the tasks from the video, the system can also estimate practice time outside of the videos.  This information is used to drive the feedback features in the tablet app (eg: progress toward weekly training time goals and the stars and checkmarks on certain buttons) and populate the trainer’s and manager’s webpage tables.  The app also records the trainees’ answers to the daily survey questions along with a timestamp.  All data is recorded on the trainee’s tablet using Android’s SQLite database and is transmitted at regular intervals to the server in a simple JSON format through an encrypted secure shell connection.  The same connection is used to transmit voice or video messages between the server and tablet.

The content—far more data than the app itself—is downloaded from the content designer’s website as an archive containing an xml file describing the structure of the content hierarchy and the data associated with each node (eg: name, description, prerequisites, etc.), and a directory tree mirroring the content hierarchy and containing the video files.

The websites supporting the other roles are implemented on top of standard Apache webservers.  Security on the server is managed with standard Linux account and file access control features.

Discussion

The EPICWheelS RCT (Giesbrecht, Miller, Eng, Mitchell, Woodgate, & Goldsmith, 2013) is a pilot study designed to determine whether feasibility indicators and primary clinical outcome measures are sufficiently robust to justify a subsequent full-scale RCT.  The EPICWheelS treatment group is compared against a control group given a collection of tablet games from Google Play (the standard source for Android apps) intended to enhance cognitive and motor skills.  The Wheelchair Skills Test version 4.1 is the primary clinical outcome measure, although user-based measures of adherence, usability and satisfaction are also collected.  Recruitment is underway in two Canadian citites (Vancouver and Winnipeg).

Clinical Evaluation Challenges

While mHealth tools bring novel capabilities to the healthcare setting, as discussed in (Kumar, et al., 2013) a variety of challenges arise when evaluating their efficacy.  The most obvious of these is that the rapid and largely uncontrollable evolution of mobile hardware and software does not match well with traditional research designs; for example, this project has been forced to transition through three different tablets, two networking devices and two major Android releases since summer 2011 as older hardware becomes unavailable.  While we do control our custom software, new Android releases inevitably perturb the user interface.   Our current compromise is to allow for updates at any time to features not visible to the trainees (eg: database management, trainer interface), to minimize updates to the trainee interface (eg: version 2 and Android updates), and to keep the content fixed (eg: the training videos and their structure).  Although live content update is an obvious and potentially useful feature, we have omitted it from the current version because of its incompatibility with standard RCT protocols.  Alternative study designs may have to be considered if we wish to take full advantage of evolving technological capabilities in the future.

Technological Limitations

The EPICWheelS study was envisioned as involving perhaps a dozen trainees at a time using a single content collection, dozens of trainees in total, a handful of trainers, and a couple of study / account managers; consequently, a number of design decisions were made that limit the scalability of the current system.  The key limitation is that trainers and managers interact with each trainee individually and manually: There are no provisions for batch account creation or data download, summarizing statistics across trainees, or self-installation of the app.  These features could be added; however, as the number of users of the system begins to grow beyond hundreds, a single central server will no longer be sufficient and it will become necessary to deploy a more advanced network infrastructure.

We considered the possibility of using the tablet’s or external sensors (eg: accelerometers, gyros, odometers) to track training sessions.  We decided not to pursue this option for initial versions because the tablet’s position on the trainee’s lap (in order to view the videos) significantly degrades the quality of the tablet’s sensor data, external sensors would complicate the trainee’s interaction with the system (eg: another battery to charge), and both sources would significantly drive up data usage (currently expensive because few trainees have broadband wireless).

Conclusion

We hope that the EPICWheelS training program implemented on the METTA platform will aid older adults to improve their MWC skills, because this population currently receives limited access to structured MWC skills training.  Development of METTA is ongoing; for example, a variety of leaf node media formats other than video (eg: text, images, webpages) are now supported.  We continue to simplify and improve the trainer, study / account manager and content designer interfaces as we receive feedback from those user groups.  Early brainstorming identified sensor data collection, games, social networking and fully remote operation (where the trainee and trainer never meet in person) as desirable features, and we hope to revisit their exclusion from the current version in the future.  Although developed concurrently with EPICWheelS, METTA is designed to be content agnostic and we welcome inquiries from groups interested in testing the system for other training programs.